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What is “Data-Based Decision Making” in Transition Planning? 


Data-based decision making refers to collecting, analyzing, and reporting data to drive 
school improvement (Dahlkemper, 2002). This can apply to any level of the school from 
individual students to the entire system. When applied to the field of secondary 
transition, it refers to using data to drive decisions to improve the in-school and post- 
school success of secondary students, as well as the systems that support those 
students. 


Why is “Data-Based Decision Making” in Transition Planning Important? 


The use of data to not only assess performance, but also to drive decisions has 
numerous benefits. Hamilton et al. (2009) identified a number of these advantages. For 
instance, data-based decision making allows educators to prioritize their time to target 
the areas most needed. It allows educators to identify strengths to be built upon and 
weaknesses to be addressed. Further, the effectiveness of interventions can be 
monitored, which allows for the refinement of interventions. This includes highly local 
interventions such as the type of feedback given to an individual student to extremely 
broad interventions such as a new district policy. Therefore, the use of data-based 
decision making in transition planning is important to ensure transition services 
provided to students are effective and efficient. 
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What Does the Literature Say About Data-Based Decision Making in 
Transition Planning? 


The following sections provide a reference and brief synopsis of articles published on the 
topic of data-based decision making. The articles are divided into three categories based 
on the focus of the article. The first category, systems level, includes articles that focus 
on data-based decision making for an educational system (e.g., school, district). The 
second category, student level, focuses on data-based decision making for students. The 
third category, NTACT, describes the resources that can be found on the NTACT website 
(http://transitionta.org/dataanalysis). 


Systems Level 


Carlson, D., Borman, G. D., & Robinson, M. (2011). A multistate district-level cluster 
randomized trial of the impact of data-driven reform on reading and 
mathematics achievement. Educational Evaluation and Policy Analysis, 33, 378- 
398. 

e In this study researchers examined the effects of a data-driven reform initiative 
at the district level. 

e Districts were randomly assigned to treatment and control conditions. In the 
experimental districts consultants from the Johns Hopkins Center for Data- 
Driven Reform in Education (CDDRE) worked to (a) implement quarterly 
benchmark assessments for students, and (b) provide extensive training on how 
to interpret the resulting data and use that interpretation to drive educational 
reform. 

e Results showed significant districtwide improvements in student mathematics 
achievement for the CDDRE districts when compared with the control districts, 
but failed to show similarly significant gains in student reading achievement. 


Cooley, V. E., Shen, J., Miller, D. S., Winograd, P. N., Rainey, J. M., Yuan, W., & Ryan, L. 
(2006). Data-based decision-making: Three state-level educational leadership 
initiatives. Educational HORIZONS, 85, 57-64. 

e This article described three cases of statewide leadership initiatives to improve 
data-based decision making. This included Ohio, New Mexico, and Michigan, all 
of which were supported by grants that were a part of the Wallace Foundation’s 
State Action for Education Leadership Project II. 

e In Ohio the project used workshops, self-paced electronic modules, and graduate 
coursework at the Ohio State University to develop a range of data-based 
decision making skills. These skills included (a) using value-added information 
and other data analysis methods to improve student achievement, (b) 
disaggregating student data, (c) monitoring yearly progress of students using 
value-added data reporting, (d) prioritizing the greatest need, (e) creating a 
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network to share information, and (f) identifying practices to raise student 
achievement. 

e New Mexico decided to target “accountability literacy,” which referred to the 
ability to interpret and analyze accountability data, using that data to support 
educational efforts across settings, and to be able to use that data to effectively 
advocate for students. 

e Michigan targeted data-based decision making at multiple levels of leadership to 
establish a coalition. These levels included the governor’s office, the department 
of education, major professional organizations, and select universities that 
prepared the majority of Michigan’s educational administrators. 


Johnson, M., & Black, B. (2012). What's going on? Analysing visual data to understand 
context-based decision-making processes. International Journal of Research & 
Method in Education, 35, 243-250. 

e This study examined how team leaders/supervisors gave feedback to examiners 
of a high stakes educational assessment. 

e Researchers observed and coded how examiners navigated the observable 
behaviors examiners exhibited during evaluation, as well as the examiners’ 
comments made when they viewed their own performance via video recordings. 

e Results showed a stable process that included three phases (i.e., Evaluation- 
Diagnosis-Feedback). First the team leaders would evaluate an examiner’s 
markings. Next they would diagnose why they thought an examiner’s mark was 
made. Last, they would provide feedback to the examiner. 


Jolivette, K., McDaniel, S. C., Sprague, J., Swain-Bradway, J., & Ennis, R. P. (2012). 
Embedding the positive behavioral interventions and supports framework into 
the complex array of practices within alternative education setttings: A 
decision-making process. Assessment for Effective Intervention. Advance online 
publication. doi: 10.1177/1534508412454450 

e Alternative education settings often use a range of curricula, interventions, and 
strategies. As such, there is no guarantee the many practices are compatible or 
implemented in an organized manner. 

e This article described a decision-making process for how the positive behavioral 
interventions and supports (PBIS) framework can be adopted in alternative 
education settings. 

e After describing the framework of PBIS, the authors described several steps to 
facilitate adoption in alternative education settings. These steps included (a) 
creating an initial list of practices, (b) analyzing the evidence supporting the 
practices, (c) keeping the evidence-based practices, (d) dividing the remaining 
practices into tiers of intervention, (e) writing formal procedures for each 
practice, (f) creating a progress monitoring plan for each tier, (g) developing a 
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“menu” of tiered support and the accompanying written procedures, and (h) 
scheduling reviews for the menu of procedures. 


Walser, T. M. (2009). Systemic data-based decision making: A systems approach for 
using data in schools. Articles on Best Practice, 6, 28-32. 

e Using the Toyota Motor Corporation as an example, this article described how 
data-based decision making can be used on a systems level to support school 
leaders and educators to promote student and school success. 

e They began by arguing for the need to focus on means, defined as internal 
operations, as opposed to ends or results. In application this means creating 
standards for the tasks educators perform and focusing on measures of those 
standards to assess systemic performance, as opposed to simply measuring 
external data sources such as achievement data. 

e The authors acknowledged the need to use external information such as student 
achievement scores in advance of interventions and for monitoring results over 
time. 


Student Level Articles 


Duffy, H. (2007). Meeting the needs of significantly struggling learners in high school: 
A look at approaches to tiered intervention. Retrieved from National High 
School Center website: 
http://www. betterhighschools.org/docs/NHSC_RTIBrief_08-02-07.pdf. 

e This article described how to implement a multi-tiered system of support, 
specifically Response to Intervention (RTI), to support students at the secondary 
level. 

e After describing the RTI model as it has been applied in elementary schools, the 
author described several implications for using RTI in secondary schools such as 
the need to monitor students who enter receiving supplemental services and the 
fact that some academic and behavioral issues may not manifest until high 
school. 

e Next various issues for RTI at the high school level (e.g., the need for screening 
and progress monitoring tools, high school appropriate interventions) were 
reviewed. 

e Finally, RTI research that was being conducted at the time of publication and 
listing relevant resources were provided. 


Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. 
(2009). Using student achievement data to support instructional decision 
making (NCEE 2009-4067). Washington, DC: National Center for Education 
Evaluation and Regional Assistance, Institute of Education Sciences, U.S. 
Department of Education. Retrieved from 
http://ies.ed.gov/ncee/wwc/publications/practiceguides/. 
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e This practice guide is intended to assist teachers and administrators in their use 
of achievement data to make appropriate instructional decisions. The authors 
focused on what they refer to as common assessments (i.e., state accountability 
tests, commercial assessments, end-of-course tests, interim district/school level 
tests). 

e The guide provides a framework for how to use these data, suggesting (a) 
multiple data sources should be used, (b) encouraging collaborative discussion 
among teachers regarding how to use the data to support student achievement, 
and (c) instructing students to monitor their own data and use it for personal 
goal setting. 


Kashima, Y., Schleich, B., & Spradlin, T. (2009). The core components of RTI: A closer 
look at evidence-based core curriculum, assessment and progress monitoring, 
and Data-Based Decision Making. Bloomington, IN: Center for Evaluation and 
Education Policy, Indiana University. 

e This report looked at three of the six core components of response to 
intervention (RTI) as described by the Indiana Department of Education. These 
included evidence-based core curriculum, assessment and progress monitoring, 
and data-based decision making. The authors described each component, 
address common concerns, and discussed application of that component. 

e Regarding data-based decision making, after the initial description the authors 
described a problem-solving process to guide data-based decision making and 
then provided an example of its application from an elementary school. 


Positive Behavioral Interventions & Supports (2015, January) Drill down tool 
evaluation brief (Evaluation brief No. 19). Eugene, OR: Eliason, B. & Morris, K. 

e This brief provided an overview of the School-Wide Information System (SWIS) 
Drill Down Tool. The tool assists in data-based decision making by filtering a 
school’s office discipline referral (ODR) data, which is collected via the SWIS. 

e The SWIS Drill Down tool offers the ability to filter core SWIS reports into 12 
categories (i.e., referral type, student/staff demographics, data/time, location, 
problem behavior, perceived motivation, others involved, action taken, 
seclusion/restraint, notes, and custom fields). 

e Anoverview of how to use the tool is provided. 


NTACT Resources 


Resources for Indicator B-13: Transition Planning and Service Compliance 
e The National Secondary Transition Technical Assistance Center (NSTTAC) 
developed an Indicator Checklist to assist states in meeting the requirements of 
Indicator 13. This resource was developed in consultation with the OSEP in 2006 
and updated it in 2009 and 2012. The checklist includes two forms. 
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o Indicator Checklist Form A is designed to meet the minimum 
requirements for collecting and reporting data on Indicator 13 of the Part 
B State Performance Plan and Annual Performance Report. It allows a 
school, district, or state to review the data for each item simultaneously 
across all postsecondary goal areas. 

o Indicator Checklist Form B is a multiple column form that meets the 
requirements for collecting and reporting data on Indicator 13. It also 
allows a school, district, or state to more deeply analyze professional 
development and program change needs by providing data on each item 
for each postsecondary goal area. 

e Asample of completed checklists and case studies are provided. 

e These resources are available at: http://transitionta.org/dataanalysis; then click 
on State Performance Plan (SPP)/Annual Performance Report (APR) Part B 
Indicator Data Collection, Analysis, and Use. 


Resources for Indicator B-14: Post-school Outcomes 
The National Post-School Outcomes Center created a number of resources related to 
Indicator B-14. These resources are organized into two categories. 

e The first category is Resources for Collecting Indicator 14 Data. This group 
contains a number of products on how to establish and ensure effective 
methods are in place for collecting and storing Indicator 14 data. Resources 
include, but are not limited to, a data collection protocol, strategies for 
contacting hard to reach youth, a response calculator, and a training guide for 
interviewers. 

e The second category is Resources for Analyzing and Using Indicator 14 Data. This 
contains products to assist with displaying, analyzing, and making decisions 
based on the collected data. Resources include, but are not limited to, a data 
display template, guide to writing suggestions, how to embed data in the 
statewide longitudinal data system, and tips for engaging parents and families. 

e These resources are available at: http://transitionta.org/dataanalysis; then click 
on State Performance Plan (SPP)/Annual Performance Report (APR) Part B 
Indicator Data Collection, Analysis, and Use. 


Core Data Tools for Dropout Prevention 
e The National Dropout Prevention Center for Students with Disabilities originally 
developed a set of Excel-based data tools to help districts and schools organize, 
examine, analyze, and share their data that impact graduation and dropout 
rates. The tools can support a school team’s work carrying out a root cause 
analysis of the factors that impact school completion rates. They also can inform 
the development and evaluation of a local intervention plan. 
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Tools include: (a) the Core Dropout Data Tool; (b) additional tools for middle and 
high school levels focused on academics, attendance, discipline, and graduation 
& dropout data; and (c) a discussion guide for use of the tools. 


These resources are available at: http://transitionta.org/dataanalysis; then click 


on Data Analysis, and Tools. 


Evaluation Toolkit 
The Evaluation Toolkit is a resource to assist educators and service providers to improve 
programs. 


The toolkit is designed to help determine what is important to your stakeholders, 
what needs to be measured to satisfy stakeholders, what is feasible to measure, 
how to measure these items, and how to report, disseminate, and use your 
evaluation findings. 

It provides specific examples for state and local teams who are developing goals 
and activities to improve transition education and services for students with 
disabilities. 

This resource is available at: http://transitionta.org/evaltoolthirdedition 


Interdisciplinary Team Planning Tool 


Ql-2 


The Interdisciplinary Team Planning Tool was designed to assist state and local 
interagency teams develop strategic plans to improve secondary transition 
education and services. 

It walks teams through a process of a needs assessment through the 
development of actionable goals including clearly delineated opportunities to 
measure progress. 

It is an online tool that includes the capacity to house data from other sources — 
including some of the data displays for Part B Indicator Data from NTACT, data 
from the VR State Plan, as well as other sources of data relevant to transition 
education and outcomes (e.g., the STEPSS tool, the QI-2, the Dropout Data Tools 
to inform the planning process). 


This resource is available at: http://transitionprogramtool.org/ 


The QI-2 is a self-assessment instrument designed to help determine the most 
critical needs within a transition program, across 7 transition domains. It was 
developed by, and is available through, the Transition Coalition. 

This resource is available at: http://transitioncoalition.org/qi-survey- 


introduction/ 


State Toolkit for Examining Post-School Success (STEPSS) 
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STEPSS is a web-based data-based decision making tool designed to support 
state departments of education in disseminating and using data related to 
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secondary transition (SPP Part B Indicators 1, 2, 13, and 14) to improve in-school 
transition programs for youth with disabilities. 

e Developed by the National Post-School Outcomes Center, the National 
Secondary Transition Technical Assistance Center, and the National Dropout 
Prevention Center for Students with Disabilities in 2013, it continues to be 
updated and maintained by NTACT. 

e A guide for using STEPSS is also provided. 
(http://transitionta.org/sites/default/files/dataanalysis/STEPSS Facilitator.pdf) 


Vocational Rehabilitation Data 
e Two resources are included under Vocational Rehabilitation Data. 
e The first is a PowerPoint presentation on how to use VR data sources for 


program improvement. The presentation comes from the National Capacity 
Building Institute in 2015. 
(http://transitionta.org/sites/default/files/dataanalysis/Using%20VR%20Data.Fa 
bian_Neubert.pdf) 

e The second resource is ExploreVR. ExploreVR a web application accessing a range 
of VR and related data to assist in planning, evaluation, and decision- 


making. (http://www.explorevr.org/) 
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